APPLICATION 
FOR 

UNITED STATES LETTERS PATENT 

Be it known that I, John B. Hey of Weston, Massachusetts, has invented a certain new 
and useful: 

METHODS FOR ACQUIRING, AND FOR DISPLAYING PREDICTIONS OF, 
SUBJECTIVE CONSUMER-APPRAISALS OF ITEMS 

of which the following is a specification: 
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Applicant: John B. Hey 

For: Methods For Acquiring, And For Displaying Predictions Of, Subjective 

Consumer- Appraisals Of Items 

CROSS REFERENCE TO RELATED APPLICATION 

This application claims benefit of Provisional application entitled "Methods for 
Acquiring, and for Displaying Predictions of, Subjective Consumer- Appraisals of Items", serial 
no. 60/443,396, filed on January 29, 2003. 

FIELD OF THE INVENTION 

The present invention relates to methods for allowing a consumer to express his 
subjective appraisal of items he has consumed, and for displaying to an item's prospective 
consumer a prediction of his subjective quantitative appraisal of the item. 

BACKGROUND OF THE INVENTION 

A given consumer's reaction to a given item may strongly depend upon the consumer's 
subjective, non-obvious tastes. For many item categories, therefore, predicting that reaction is 
problematic, even when detailed objective item-descriptions are available beforehand. Common 
examples of such categories include movies, books, and music. Still, the many consequences of 
wrongly chosen items make reliable individual predictions highly desirable. E.g., when 
informed by such predictions, trips to the multiplex now find the "better" movies, and fewer 
novels disappoint. 

Since 1985, some popularity has been afforded a computer-oriented approach to 
predicting consumers' subjective reactions to items. With that approach, each consumer rates 
items already consumed, by placing each item on a fixed scale ranging from, e.g., "10" to "1", or 
"good" to "bad". The recorded set of the consumer's ratings comprises the indication of that 
particular consumer's tastes, which the computer then analyzes vs. the rating-sets of other 
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consumers, in order to predict the original consumer's probable rating of a prospective item. 
(Many kinds of analyses are possible. For example, if two consumers have rated many items in 
common, and furthermore have rated each item similarly , then the computer might consider 
each consumer's additional item-ratings to strongly predict the other consumer's prospective 
5 rating of those items.) The predicted rating is typically displayed as a value on the same scale. 

With such computer-oriented predictors, the acquisition of a consumer's subjective item- 
rating comprises the consumer's direct assignment of a scalar value to the item in question, from 
a static scale of possible values (or range of values). 

One weakness of the static scale is that many consumers can't readily resolve it to their 
10 own innate sense of item quality (even if the scale is marked with verbal "hints", e.g., Good, 

Fair, Poor). And even consumers who do feel comfortable with such a scale often misuse it. For 
example, they typically mis-assign their first items, by not placing them on the scale where 
they'll later wish they had, after rating additional items. Then, the consumer must either re-rate 
some items or else be resigned to a continuing misuse of the scale. 
1 5 Another weakness of the static scale arises from consumers' tendency to inflate 

(particularly) its outlying values by rating recently consumed items "enthusiastically", i.e., overly 
high or low. A static scale panders to such abuse, by offering a seemingly inexhaustible supply 
of all scale values. (This is somewhat analogous to the classroom practice of "grade inflation".) 

A fiirther weakness of the static scale arises because the subjective "feeling" that a 
20 consumer associates with (say) the middle value of the scale readily varies over weeks and 
months, thereby compromising his appraisals' accuracy as a consistent representation of his 
tastes. 



{ H:\PA\CORP\ 1 6954\00008VA0684489. DOC } 



Still another weakness of the static scale is especially prominent in the movie 
recommendation system proffered by online DVD-renter, Netflix, Inc., whose consumers rate 
movies (DVDs) on a scale of merely five points. (Other rating systems use as many as a 
hundred.) Presumably, Netflix intends to unburden its consumers of having to discern subtle 
5 differences in their subjective enjoyment of movies. But, regardless of how unburdened the 
consumer may feel, the need for subtle discernment nevertheless remains. For example, when 
the consumer feels that a movie merits the unavailable appraisal of "four-and-a half 1 , not only 
must a reason be fabricated for awarding, say, the "five" rather than the "four", but the choice is 
quite critical because it spans a full 20% of the five-point appraisal scale. In defense of Netflix's 
10 system, meanwhile, note that a rating scale of, say, a hundred possible values (or worse, a 
"continuous" scale) obviously overwhelms many consumers. 

Importantly, weaknesses such as the above not only compromise consumers' ability to 
rate items accurately, but also degrade the accuracy of a computer's predictions subsequently 
based upon those ratings. 
15 SUMMARY OF THE INVENTION 

Taking the above into consideration, an object of the present invention is the acquisition, 
in a manner convenient to consumers, of a consumer's accurate subjective appraisal of a 
consumed item. 

Another object of the present invention is the display of a consumer's predicted appraisal 
20 of an item, in a manner intuitively comprehensible to the consumer. 

For systems that collect and predict subjective item-appraisals, the present invention 
provides a new means for a consumer to arrive at and to specify to the system each appraisal, as 
well as a new and related means by which the system may present its predictions to the 
consumer. 
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In one illustrative embodiment, the consumer, using his computer system's keyboard and 
monitor, selects for appraisal several items he's already consumed. He then rearranges a list of 
the selected items to reflect his relative subjective appraisal of each item, e.g., ordering them 
from "best" to "worst". Later, when he wants to appraise an additional item, he retrieves his 
5 ordered list and inserts the new item at a position reflecting his appraisal. If two or more items 
are "equal" in his view, he may place them at effectively the same list position. 

When the system predicts the consumer's appraisal of an item (usually an item he hasn't 
yet consumed), the prediction is shown as a position (or range of positions) on his list. The 
present invention also includes a method for displaying an item's prediction as a "graded" range, 
10 which depicts an increasingly narrow, and decreasingly reliable, value range for the predicted 
appraisal. 

A variation of the preceding embodiment allows the consumer to adjust the space 
between adjacent items on his list, to reflect the degree of difference in their appraised 
"goodness". 

1 5 Some advantages of the present invention are already apparent: 

The consumer need never connect his sense of an item's worth to a number on an 
arbitrary scale. Rather, he merely decides which of (say) two items he liked more ...an intuitive 
task he can easily perform with consistent accuracy (unlike the task of quantifying his enjoyment 
as an absolute value). 

20 He can express his perceived spectrum of item-quality as coarsely or finely as he 

chooses, because, e.g., he can always insert a new item at an "in-between" position. He need 
never attempt a prescient apportionment of a rating scale, because each newly entered item can 
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always be inserted simply and exactly in relation to his already entered items, thus easily 
accommodating his opinions' full breadth and granularity as they evolve over time. 

When appraising a new item, he is automatically made aware of the "meaning" of 
whatever list position he chooses. E.g., he's likely to consider carefully whether he truly enjoyed 
5 the new item more than its next-lower neighbor ...and the answer is readily forthcoming, 
requiring merely that he remember the other item well enough to realize which one he liked 
more. 

(Note that the preceding advantages facilitate the consumer's accuracy in presenting 
appraisals to the system.) 

10 Analogous advantages apply to the system's display of a predicted item-appraisal as a list 

position. For example, the consumer grasps a prediction's true meaning through his memories of 
its adjacent list items, i.e., items he has already appraised and "understands". Such intuitive 
comfort with a numerical scale arrives only over time, if ever. Furthermore, because the list 
directly reflects the consumer's individual appraisal tendencies (e.g., with respect to granularity, 
15 or thresholds of item-worthiness), an item's predicted list position means even more to an 
individual consumer, by resonating more closely with his internal tastes. 

The above and other objects, features, and advantages of the present invention will 
become apparent from the following description, taken in conjunction with the accompanying 
drawings which illustrate preferred embodiments of the present invention by way of example. 
20 BRIEF DESCRIPTION OF THE DRAWINGS 

Figs. 1 A and IB show an example of a prior art arbitrary scale of values; 
Fig. 2 is a flow chart of a typical opinion-predicting system that can be used with the 
invention; 
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Figs. 3A-3C depict a list of previously-appraised movies and a user's insertion of a 
newly-appraised movie into the list, according to the invention; 

Figs. 4A and 4B depict a list of previously-appraised movies and two alternative manners 
in which a prediction relating to another movie can be presented to the user, according to the 
5 invention; 

Figs. 5A-5D depict a list of previously-appraised movies and different embodiments of a 
user's insertion of a newly-appraised movie into the list, according to the invention; and 

Figs. 6A-6E depict a list of previously-appraised movies and several alternative manners 
in which a prediction relating to another movie can be presented to the user, according to the 
10 invention. 

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 

Prior to the present invention, the consumer appraises a movie he's watched by 
effectively assigning it a position on an arbitrary scale of values, e.g., 1-10, Poor-Fair-Good- 
Excellent, or Thumbs Up-Level-or-Down, as exemplified in FIGS. 1 A and IB. The position he 

15 assigns is supposed to reflect his subjective appraisal of the movie. He therefore must choose, 
and somewhat memorize, how to apportion the scale with respect to his own past (and future) 
personal movie experiences. And he must choose well because, e.g., ill-chosen assignments of 
his initially entered movies can effectively confine his later movie ratings (and resultant 
predictions) to a tiny segment of the scale. 

20 Certain preferred embodiments of the present invention will now be described in 

conjunction with the accompanying drawings. 

FIG. 2 illustrates a typical opinion-predicting system, in the item domain of movies. In 
this environment, a consumer identifies a movie of interest 60. If he has already watched the 
movie 61, he enters his subjective appraisal of it 62. If, however, he hasn't watched it, a 
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prediction of how high his appraisal of the movie would be is computed 64, and displayed to him 
63. The present invention concerns how he enters his appraisals 62, and how predictions are 
displayed to him 63. 

Note that the prediction methodology 64 may range from the trivial to the intricate, and 
5 that the present invention is independent of any particular such methodology. (A primitive 

methodology might notice, for example, that most consumers who've rated both movie-A and -B 
have furthermore given them similar ratings; so, a new consumer who rates movie-A low would 
be predicted to rate movie-B low, too. Naturally, much more sophisticated methodologies are 
possible.) 

10 First Example Preferred Embodiment of the Invention 

Using a computer screen and keyboard, a consumer wishes to enter his personal appraisal 
of movies he's recently watched (or books he's recently read, or music he's recently heard, for 
example). By doing so, he intends to inform an underlying computer program of his personal 
taste in movies, so that he can later receive from the program accurate predictions of his 

15 appraisals of movies he hasn't yet watched. Because such predictions are (typically) based upon 
an inter-personal analysis of many such appraisals entered by many such consumers, the 
consumer wants his entries to reflect accurately his true movie tastes, so that the program's 
resulting predictions for him will be likewise accurate and reliable. 

For this example embodiment, consider a consumer who has already entered his relative 

20 appraisals of several movies he's seen, and who now wishes to enter a new movie appraisal. 
Accordingly, he is shown his current list, ordered "best"-to- fl worst" (FIG. 3A). After some 
reflection, he realizes that he enjoyed the new movie (entitled "Love's Jail") less than the second 
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listed movie 20, but more than the third listed movie 21. He therefore inserts the new movie 22 
into the list, placing it into the "gap" between those two movies (FIG. 3B). 

Alternatively, he may insert the new movie exactly upon an already listed movie (FIG. 
3C), thereby equating them. The consumer may at any time rearrange the movies comprising his 
5 current list, thereby revising his earlier entered appraisals of them. 

In return for entering his movie appraisals, the consumer hopes ultimately to receive a 
meaningful prediction (as calculated by a computer program) of his opinion of one or more 
movies he hasn't yet seen. The present invention presents such a prediction using an ordered list 
comprising movies he has previously entered. As exemplified in FIG. 4A, the consumer's 

10 enjoyment of his unseen movie 32 is predicted to lie between that of his already seen-and- 

entered movies 30 and 31. The prediction may alternatively encompass a range of positions; the 
example in FIG. 4B predicts (using a bracket or another icon) that the consumer's enjoyment 
would lie somewhere between exceeding movie 33 to equaling movie 34. 

Second Example Preferred Embodiment of the Invention 

15 In a further embodiment, the scale (comprising the list of the consumer's previously 

entered movies, ordered by his enjoyment of them) indicates also the relative enjoyment 
"distance" between adjacent movies. For example, in FIG. 5A the consumer enjoyed movie 40 
much more than movie 41, and thus the gap between them is large. Meanwhile, he enjoyed 
movie 41 only slightly more than movie 42, and thus the gap between them is small. In such an 

20 embodiment, a consumer may accordingly refine his opinion of an entered movie. For example, 
in FIG. 5B he has judged the new movie 49 to be only slightly inferior to the better movie 48; 
thus his placement of the new movie leaves movies 48 and 49 almost touching. (For example, a 



{H:\PA\CORP\l 6954\0O0O8\A0684489. DOC} 



typical computer drag-and-drop interface permits the consumer to drag movie-title 49 upwards 
until it nearly touches movie-title 48.) 

Similarly, this embodiment displays a prediction by placing it appropriately nearer the 
top or bottom of its gap. For example, in FIG. 5C because the consumer's enjoyment level is 
5 predicted to be rather nearer the bottom movie of its gap, the prediction 43 is placed 
proportionally further down the gap. 

More generally, in such embodiments the gap shown between two movies reflects the 
consumer's relative enjoyment "distance" between them. However, note that not merely actual 
space, but virtually any informational characteristic can be employed to convey a gap's size. 

10 FIG. 5D illustrates an example embodiment that does not vary shown distances, using instead the 
presence or absence of a connector to liken adjacent movies. The movie scale shown indicates 
that the consumer enjoyed movie 44 only marginally less than movie above (hence the connector 
at 45), but distinctly more than the movie below (hence no connector at 47), and exactly as much 
as the (sideways contiguous) movie 46. 

15 Third Example Preferred Embodiment of the Invention 

The present invention includes also a method of displaying a prediction on practically 
any ordered scale (including scales claimed by the present invention). According to this method, 
the consumer is shown a "graded" range of values (i.e., with each value graded according to its 
predicted likelihood) comprising the prediction of his enjoyment of the movie. 

20 For example, in FIG. 6A the full prediction comprises a high confidence ("tier" 50 with 

relatively large arrows at the ends) that the consumer would give the movie 52 an appraisal 
between very-high and moderately-low, and a lower confidence (tier 51 with smaller arrows at 
the ends) that his appraisal will lie toward the upper end of that range. Alternatively, such tiers 
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of probable ranges can be smoothed to an approximation having a different appearance, for 
example as illustrated by FIG. 6B, which approximates the combined prediction shown in FIG. 
6A. 

By way of further example, the prediction may comprise a "scatter plot" indicating the 
relative probability assigned to one or more scale areas, as exemplified in FIG. 6C. Such a 
scatter-plot prediction may likewise be approximated in a smoothed display as illustrated by FIG. 
6D. 

FIG. 6E exemplifies a graded prediction applied to an ordinary fixed and static scale of 
possible appraisal values (cf FIG 6A). 

Note that the display or presentation of a list can alternatively occur on a document 
printed by the computer system. 

What is claimed is: 

Other embodiments will occur to those skilled in the art and are within the following 
claims: 

What is claimed is: 
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